In human language we constantly use context to make sense of each others utterences and to shorten our own. This applies at a syntactic level, for example the use of 'This' at the begining of this sentecne rleias on the context that we are talking about "context in language". It also applies at semantic level, for example, if you were asked "how far is it to New York?", you would normally assume this was relatve the current location of the speaker. In natural language processing we need to use context to disambiguate words and phrases and also to establish higher level meaning. Transformer models use previous context in the text to help decode the most likely next word; this is how prompt engineering in ChatGPT and other LLM systems work.
Used on page 21